Return-oriented programming (ROP) exploits are an increasingly common form of malicious software (malware) that may circumvent certain defenses that mark locations of memory as non-executable. An ROP exploit works by stringing together a large number of existing segments of executable code that each end with a “return” instruction (known as gadgets). Each ROP gadget is typically short, and typically does not correspond to an existing procedure or even an existing instruction boundary in the executable code. The attacker constructs a malicious stack including a series of return addresses pointing to the desired sequence of gadgets. The ROP exploit is performed by causing the processor of the computer to execute software using the malicious stack instead of the legitimate system stack. For example, the malicious stack may be introduced by smashing the stack, using a buffer overflow exploit, pivoting to a new stack, or otherwise corrupting the system stack. Jump-oriented programming (JOP) and call-oriented programming (COP) exploits are similar, but target gadgets that begin or end with an indirect jump instruction or a call instruction, respectively, rather than a return instruction.
Certain ROP exploits may be prevented by maintaining a “shadow stack” in parallel with the ordinary system stack (also called the “legacy stack”). The shadow stack maintains a copy of the legacy stack in memory inaccessible to ordinary software, and may be used to determine if the legacy stack has been tampered with by malware. Other measures available to help prevent ROP, JOP, or other control flow exploits may include inserting ENDBRANCH instructions into binaries to specifically mark allowed targets for indirect branches or jumps.